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Vlevpoints on the role of educational research and 
evaluation in the 1960*s is presented* Semarks focus on four major 
themes: (1) state level research and evaluation has progressed from a 
p&ssiver primarily reporting rcle tc an activer occasionally advocacy 
role; (2) the educational questions and the question askers have 
tecome more sophisticated and will increase in sophistication; (3) 
primary consumers of state level research and evaluation have changed 
from other researchers and evaluators to include diverse publics 
(e*g*r legislatorsr state budget analystsr lobbyists from special 
interest qroupsr TV and newspaper reportersr professional 
crganiza ticns r and other state government agencies); and (4) the 
state education agency has a unique rale to play in producing and 
consuming research and evaluation* It is forecasted that as the 
decade of the 1960*s disappearsr state education agencies vill have 
enhanced their current capabilities in research and evaluation to 
qiiide action cf state policy makersr to provide better bases for 
decisions and to improve educational programs fcr children* (BL) 
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It's of historical interest to be in Massachusetts today to discuss what 
state level research and evaluation should be for the decade of the 'BO's. 
It was in Massachusetts in 1837 that a State Board of Education was first 
established. In 1900^ there were less than 200 professionals in all state 
departments of education; today, there are approximately 2«000 professionals 
in state education agencies who are engaged in research, evaluation, 
planning and assessment alone* 

My remarks center on four major themes; 

Themes 

(1) State level research and evaluation has progressed from a passive, 
primarily reporting role to an active, occasionally advocacy role. 

(2) The educational questions and the question askers have become more 
sophisticated and will increase in sophistication* 

(3) Primary consumers of state level research" and evaluation have 
changed from other researchers and evaluators to include diverse publics. 

(4) The state education agency has a unique role to play in producing 
and consuming research and evaluation. 



*This paper includes the insights and assistance of Sally Pancrazio, Tom 
Kerins, and Connie Wise of the Illinois Board staff. 
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As each theme is identified. Til give further detail and examples, 
stemming from my five years' experiences as the 111 Inois State 
Superintendent* As an aside, it was Interesting for me to see Bill 
Schubert's review of institutional contributions to the AERA annual program 
as an indicator of research productivity** For the period in which I was 
State Superintendent, the 111 inois SEA ranked first* This was the first 
productivity review which counted SEA contributions* What Schubert's review 
shows is that SEAs are engaging in research and evaluation activities which 
are being accepted by the traditional research community* 

Theme I* State 1 evel research and eval uati on has progressed from a 
passive^ primarily reporting role to an active, occasionally advocacy role * 

Historically, the forerunner of any research and evaluation was the mere 
collection of statistics—a task apparently endemic to governmental agencies* 

From the 60' s through the late 70' s, SEA evaluation was basically 
passive in nature* Questionnaires and surveys were disseminated to local 
educational agencies requesting data; and very little, if any, feedback was 
provided to those who submitted the information, nor did those individuals 
have much input into the content of evaluation* Data were collected to 
respond to specific requirements from either^'the federal government or state 
General Assembly* 

Also> during this time, program evaluation was often confused with 
compliance monitoring* Although there is some overlap* results of 
compl i ance moni tori ng acti vi ti es do not often coi nci de wi th judgments 
relevant to program worth or impact* 



* William Schubert, (University of II linois-Circle Campus), "Institutional 
Contributions to the AERA Annual Program as an Indicator of Research 
Productivity/* Educational Researcher, July-August, 1979* 



Beginning with the late 70's> evaluation has taken a more active role* 
SEAs now participate as part of the CCSSO/CEIS subconwiittee in providing 
input to the federal government on evaluation designs and instrimients* LEAs 
have an opportunity to provide SEAs with their perspective as to what is 
important^ what should be required^ and how the results might be useful on 
the local level* The utility of evaluation results has also been redefined;^ 
the emphasis now is on using results in quantifying decision making^ 
planning^ and policy formulation* 

Through increased staff expertise in SEAs as well as statistical 
capacity building grants from NCES> there has been the development of 
statewide statistical data bases* These data bases--on pupils^ fi nance > 
personnel > programs > and facilities—now allow projections > simulations > 
comparisons^ and trend analyses which quantify recommendations for action by 
SEAs* 

In the past> for example^ the Illinois SEA reported pupil enrollment 
statistics as part of its reporting mandate to the Governor* Now> with the 
increased need for information and better analyses > projections of State 
enrollment data are made based on a series of assumptions about State 
population trends and future live birth data* These projections are used in 
long range planning by the Illinois State Board of Education* 

A second example—this time from the school finance area — * In the 
past> finance statistics were gathered^ reported in descriptive reports^ and 
disseminated back to the di stricts* Now> in response to EEO litigation^ 
SEAs are examining the impact of their school finance formulas on equity and 
fiscal neutrality principles* Aided from Section 842 federal funds> the 
Illinois SEA has conducted nimierous studies^ using the finance data base> to 
simulate and assess the impact of alternative funding formulats on equity 
and the distribution of state aid to districts* 
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Theme 2: The educational questions and the question askers have bocome 
more sophisticated and will Increase 1n sophistication * 

Public scrutiny of the schools has Intensified in Illinois as 
elsewhere* Clearly, education has produced better critics* The questions 
being asked—and the types of people asking the quest1ons--are wordly and 
percepti ve * A f avori te quest 1 on asked by the 11 1 i noi s publ i c and 
legislature is: "With enrollments declining, why are educational costs 
Increasing?" The denand for evidence and the quantitative justification for 
action has intensified at all levels* 

This growing sophistication is most visible In our homes— where all 
family members are bombarded by the offspring of computer technology— pong 
games; hand held computer games; the replacement of the slide rule and bulky 
adding machines with pocket-sized Texas Instruments* Home computers—ala 
Radio Shack and Apple Computer Systems--made their debut this last Christmas 
season* Computer programmi ng and i nf ormation processi ng is becoming 
integral to American life. The*80*s will see a further technological 
explosion as the publ 1c gets use to producing and understanding data 
displays* They'll expect a fast turn around time on the production of data, 
too; 

Political polls--the thermometers of public opinion—are easily 
understood by the public* The concept of probability sampling—which is the 
essence of polling— will be taken for granted as people come to expect to be 
surveyed on various issues* These, too, have aided in the public's 
developing sophistication about education questions and the demand for data* 

The point is that there is a push for better information about the 
schools — information which addresses the effects of various programs under 
various levels of funding: 

6 



"Which shows what will happen if various conditions are changed; 

—which describes the status of educational progress of our students and 
quantifies their needs. 

... The sophistication of the question askers--honed by their own, hands-on 
technological experience—requires that research and evaluation results be 
readily translatable into policy or action considerations* These 
sophisticates won't accept only a numeric story of percentages, 
correlations, standard devisions, and T scores* Reports must be written in 
such a way as to be technically sound and defensible yet educationally 
decipherable. 

Further, these new questions askers won't wait for glossy reports to be 
printed in order to get their answers* The timing of releasing research and 
evaluation results is critical to SEA credibility* One of the various 
ecological like disasters of the 60s and 70s hinting education is 
"literature pollution"-- the publ i cation of dated, ungeneral izable, and/or 
overly cautious findings* 

It is a myth that research and evaluation results will lead directly to 
action* It is the use and quality of data-based alternative strategies in 
the discourse and debate about issues that will enhance the state level 
decision making process* 

Theme 3* The primary consumers of state level research and evaluation 
are no longer merely other researchers and evaluators * 

Consumers of research and evaluation for the '80' s decade will be 
pluralistic and possibly adversarial— as various groups compete for their 
slice of the educational pie--services* Current consumers include: 

(1) Legislators and their staff need information upon which to make 
recommendations on whether a program should continue and whether current 
legislation is meeting the needs of the constituents. 



(2} State budget analysts need to know how much Is being spent^ or 
should be spent for educational programs* and the fiscal Impact of that 
program on the State's financial resources* ^ 

(3) Lobbyists frOTi special interest groups want results which support 
their respective causes* 

(4) TV and and newspaper reporters want information which is '*hot>** 
controversi al » and of significant Interest to their viewers and readers* 

(5) Professional organizations want data that describe the state-of-the 
art of and for their members* 

( 6) Other State government agenci es > parti cul arl y human servi ce 
agencies^ want access to school-based information which aids in their own 
program planning^ budgeting, policy development* 

The scarce economic resources for the decade of the *80*s require more 
collaboration than ever before — and no where Is this more viable than the 
sharing of information* 

Some examples of collaborative consumption of research and evaluation: 
In Illinois, 3 teacher unions and 3 management associations collaborated 
with the SEA to develop a ''Teacher/Board Collective Bargaining Information 
System*** Based on statistical data collected by various agency units, a 
computerized system was developed by which these diverse groups would have 
access to timely and accurate information about salary schedules, salary 
policies, contractual provisions, and fiscal data* These data were 
determined to be relevant to the collective bargaining process as perceived 
by parties on both sides of the negotiation table* The actual contract 
provisions are identified and microflched so that parties can review actual 
negotiated language and determine the extent to which that provision is 
present in Illinois contracts* The process of selecting which elements to 
be i ncl uded i n the system was in 1 tsel f a negoti ated agreement but the 
process represents a pluralistic perspective* 



Every year> the niinois SEA conducts a Teacher Supply and Demand Study 
in collaboration with the 111 inots teacher education institutions and the 
Illinois Association of School , College^ and University Staffing* These 
data are also made available annually to the Illinois State Teacher 
Certification Board and the Joint Education Commi ttee--the constitutionally 
mandated board with representatives from the 111 inoi s Board of Higher 
Education and the Illinois State Board of Education* 

This year, the Illinois SEA collaborated with the IHBE and the Illinois 
State Vocational Advisory Council to augment the NCES federally funded 
national study, "H*S, apd^ Beyond," a study of Illinois seniors and 
sophomores, in order to obtain state general izable data about Illinois 
students* 

For the past several years, the Illinois SEA collaborated with the 
Illinois Department of Public Health (IDPH) on assessing the status of 
immunization of school age children* The Illinois SEA collects the 
statistics on polio, DTP, measles—and as a result of the school and 
di strict analyses, IDPH can target immunization programs in specific areas 
where epidemics are most likely* IDPH is the source for our obtaining live 
birth data which Is the basis for our school enrollment projections* 

The Illinois SEA collaborated with the Illinois Assocation of 
Supervi sion and Curriculum Development to conduct a "Census of Secondary 
School Enrollments, Offerings, and Co-Curricular activities*" Out of these 
data came many reports — profiles and curricular offerings by school; special 
reports on subject areas (i.e*, foreign language, English, social studies) 
and after-school activities. Since the data were reported by gender, sex 
equity advocates found the data useful. Policy analysts concerned about the 
relationship of school size and community type to curricular offerings also 
found useful data. 



The personnel data base finds users at all levels In the educational 
arena* The data system is individual-specific for all certificated staff in 
Illinois schools--it has the name, salary, sex, race, position, certificate, 
birth date, years experience, and teacher institution granting first and 
last degree* The Illiois Women Administrators examine annually the status 
of women administrators* Teacher education institutions obtain a listing of 
their "graduates each year*' to be used for follov/up purposes and to assess 
the impact on the Illinois teaching force made by that institution* 
Inservice education can be quantified by data from such a system* 

Legislators qoncerned about the impact that proposed finance reform will 
have on their respisctive school districts request and obtain ump-teen sets 
of computer printouts showing the relative distribution of the State aid* 

CETA agencies concerned with youth employment request and receive drop 
out statistics by prime sponsor and balance of state regions* 

The Boys and Girls Scouts of America— and the military— use enrollment 
statistics for planning and recruiting purposes* The Illinois SEA uses 
enrollment data iirsuediately after collection in the fall to assess the 
status of school desegregation efforts* 

The Illinois SEA was required to conduct a third pary evaluation of the 
state funded Transitional Bilingual Program* A major question being asked 
was whether students in the state funded Transitional Bilingual Education 
Program were learning English adequately/ and whether they were transferring 
into English-speaking classrooms* The state office encouraged the General 
Assembly to fund an external evaluation.; o^ the program during the 1978-79 
school year* The evaluation was authorized as a line item in the agency 
budget. The direct impact of the evaluation/ a favofable one, resulted in 
increased funding for the program. 
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An advisory panel composed of local educators » advocate groups » 
legislators and their staffs met bimonthly to learn about the progress of 
the student and to advise the consultants* The SEA used Its position to 
insure that formative communication with all the interested parties would be 
maintained^^that these decision makers would be aware and understand as much 
about the results of the study as possible prior to its completion* 

We are including legislative staff and the media in our review of the 
state assessment data* By including these groups from the beginning^ they 
become aware of policy implications very early; and they force technicians 
to present results and interpretations simply* 

These are just some of the many examples of how basic statistics and 
research and evaluation results from the Illinois SEA are being requested 
and used by diverse groups* 

Theme 4: The state education agency has a unique role in research and 
evaluation * 

Staff expertise and professionalism has unquestionably increased in 
state education agencies* In the Illinois SEA» for example » under my 
administration^ the number of doctorates increased while the number of 
personnel decreased* With the changes in SEAs from political » patronage 
positions to professional positions under appointed State boards and/or 
Superintendents > came increasingly well trained staff--staff from the 
business^ legale and educational arenas* 

The SEA has a unique—and well qualified—role to play relative to 
research and evaluation* The state education agency has the statuatory 
authorization to develop comprehensive data bases and has the capability to 
provide access to these data by others external to the SEA* The SEA can be 
a hub for an information dissemination network within the state* Once 
credibility is established^ the SEA is in the best position to be the link 
between the LEAs and universities and the LEAs and the federal government* 
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The Illinois State Board of Education has a data release policy and 
commitment to provide Information services to the Illinois public and 
education Interest groups* The SEA must demonstrate that It can collect^ 
aggregate^ analyze^ and most importantly^ report information back to LEAs in 
such a way as to aid appropriate comparisons* Too often> the image of the 
SEA is that of a giant data warehouse—all consumption but little production* 

Ironically^ at the time there is a push for better data there is also a 
push for a reduction in the paperwork burden. In response to this 
situation^ the Illinois State Board of Education established an Office of 
Data Management (ODMA) which is responsible for facilitating the activities 
of a Data Coordinating Council , The DCC and ODMA resemble in nature to 
FEDAC/OMB on the federal level* Careful scrutiny is given to every data 
collection form which comes before the Council, Sponsors of data collection 
forms are required to justify each item queried^ how the data will be used> 
and what the burden will be. Consideration must also be given to whether or 
not the same data are collected elsewhere and if so> whether retrievable* 
Estimates of burden VS utility are made in deciding to approve such data 
requests. These activities include all research and evaluation studies 
which require data not already in the data bases. It is clear that data and 
information are resources — like dollars—which must be managed appropriately* 

As the decade of the '80's disappears^ SEAs will have enhanced their 
current capjibi Titles in research and evaluation to guide action of state 
policy makers > to provide better bases for decisions > and potential ly> to 
improve educational programs for children* 
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